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(57) Abstract 

A data warehouse system and method. The data warehouse includes a memory and a processor. The memory includes a database 
having a plurality of data entries. The processor includes a cache and an override engine, wherein the cache includes a subset of the 
plurality of data entries and wherein the override engine extracts data from the cache for viewing by a user, modifies the data in response 
to one or more user commands and saves the user commands to a file for later application to the database. 
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WO 99/57658 PCT/US99/09633 
SYSTEM AND METHOD FOR UPDATING A MULTI-DIMENSIONAL DATABASE 



Backgroun d of the Invention 

Field of the Invention 

The present invention relates generally to database management, and more 
particularly to a system and method for manipulating facts in multi-dimensional databases. 

Background Information 

Business decision-makers operating in today's rapidly changing business environment 
need answers to a host of questions that directly impact their ability to compete in the 
marketplace. To manage and use information competitively, many companies are 
establishing decision support systems built around a data warehouse. A data warehouse 
stores a company's operational and historical data in an integrated relational database for 
decision support applications, business data access and reporting. Decision support systems 
access such databases to analyze and summarize corporate performance. 

Data warehouses employ relational database management systems that use a language 
such as SQL to retrieve rows and columns of numeric data. The systems may also permit 
access to textual files such as documents. Data may be accessed directly via user-generated 
SQL commands, or indirectly, via an interface which generates the desired SQL commands. 
Applications such as Business Objects from Business Objects S.A., France, 
(httpr/Avww.businessobjecfs.com), Forest and Trees from Platinum Technology Inc., 
Oakbrook Terrace, IL, (http:/^rv^.p^ and Pilot's Lightship from Pilot Software 

Inc., Cambridge, MA^ 0ittp://i^^ are typical of off-the-shy f applications 

which use browse windows under the) ^ntrof pf end-users to generate the SQL code needed 
to analyze the data in the data warehouse. 

This approach, however, runs into significant performance problems associated with 
PC arid network limitations. Queries generate by inexperienced users pan dominate and 
crash the database, or cause excessive network congestion. In addition, there is no 
mechanism for shifting large processes so they execute during off-peak hours. 

OLAP (OnLine Analytic Processing) technology, also called multidimensional 
analysis can also be used to access the data warehouse relational database. Multidimensional 
analysis systems have been available for over 1 5 years, first on mainframes and then on 
client/servers. Under multidimensional analysis, data is divided into the dimensions and facts 
needed to manage the business. Dimensions for marketing applications may include 
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products, markets, distribution channels and time periods. The dimensions are used to 
reference specific points in a database. What that point represents is called a fact. As 
examples, units sold, revenue, and price are all facts. Dimensions are further described by 
attributes, such as size, flavor, location or fiscal year. Attributes also describe hierarchies 

5 within a dimension, even overlapping and inconsistent hierarchies. These hierarchies 

determine the, vertical relationships within a dimension. For example, in a Period dimension, 
a standard hierarchy is year quarter -> month -> week T > day. By defining these 
hierarchies, it becomes joossible for OL^applip^oii^to automatically shift their 'view* up 
or down a hierarchy. This is commonly referred to as/drilling' within this application space, 

10 an example of this would be shifting an annual report's total 199j7 data down to view the 
individual quarter's numbers., : ; 5 i: . - , ... r :.,j.,\, , ; ,. . t .;; . v . : 

; Multidimensional analysis allows users to. select, summarize, calculate, format and 
report by dimeraiqns and by attributes* wtl^ v ^mraripns. . 1\ can be used to support virtually 
any timeTseries decision.support appUcatipnmcludmg ^or&ig, analysis, forecasting and 

15 budgeting.- . : . . ..... MJ . iv K,; b . r . ^ 

To be useful, decision support systems must support analysis based not only on 
historical data but also on projections, for, future activities. For instance, marketing may 
project sales for the next three months^ These fig^^ t may .th^n be introduced into a model 
used to tune manufacturing output over that period of time. To date, such analysis has been 

20 performed using multi-dimensional databases hafymg fixed locations. What is needed is a 

system and method of extracting and modifying .infpm existing database which 

can be applied to a relation^ database in ordpr toJree the ^ organization firom the space 
limitations of multidimOTsio^ a system and 

method capable of creating reports not only based on existing information but also on . 

25 projections of future activities.* v , :: - i: 

Summary of the Inventio n 
The present invention is a data warehouse system and method. The data warehouse 
includes a memory and a processor. The memory includes a database having a plurality of 
data entries. The processor includes a cache and an override engine, wherein the cache 
30 includes a subset of the plurality of data entries and wherein the override engine extracts data 
from the cache; for vigvring by a user, modifies the data in response to one or more user . 
commands and saves the user commands to a file for later application to the database. 
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According to another aspect of the present invention, a data warehouse is described. 
The data warehouse includes a plurality of workstations connected to a memory by a server. 
The memory includes a database having a plurality of data entries. The server includes a 
cache and an override engine, wherein the server operates m response to user commands to 

-5 store a subset of the .plurality of data entries in the cache, wherein the override engine extracts 
data from the cache and sends the data extracted frbhi the cache to the client workstation for 
viewing by a user, modifies the data m response to one or more 'of the user commands, saves 
the user commands to a file and operates m re^ 
database based on the user commands stbted m the file! 

10 A(xording to yet ahomer aspect 

from a data warehouse is described in which the steps are providing a server arid a mdmory 
device, storing a database in the nieniory'd^ of 
data entries, extracting a subset of dam^enWieWn^nVihe database, sform^^^bse^Vifda^B'' 
entries on foe^erver, modifying m^ 

1 5 commands, reading data from the modified data entries stored on the server and displaying 

, ..... ".--"f si 1 ,; : : i/ fiiH-J -;-;;rn v v. .v 

the data to the user. 

; According to y et another aspect of the' present mVention, a method of forecasting 
based oh data in a data warehouse wdes^b^m^^wh^-me'st^-are providing a server and a 
niemory device, storing ;'a" database in me membry device; wherein the daiabase includes a 

20 plurality of data entries, extracting a subset of data entries fronl'me database,- storing the 

subset of da^ entries on the 'server; ; n^fymg^ed^ me server m response 

to user conrniandsrstbrihg the tisef commahdsl reading data fromf the modified data entries 
stored on the server, liispiaying me 1 data^m^er and -moaMf^g me database based on the 
stored user commands. ''*' ' * ! '■' ' 

25 According to yet another aspect of the present invention, a riiethod of increasing the 

speed in which changes to a relational database are reflected back to the user is described in 
which the steps are extracting a subset of data from the database, wherein the step of 
extracting includes the steps of displaying a representation of the subset of data to the user 
and storing the subset of data in a cache, receiving a data modification command, storing the 

30 data modification command in a file and applying the data modification command against the 
subset of data stored in the cache, wherein'the step of applying the data modification 
command includes the step of modifying 1 the subset of data tb reflect application of the data 
modification command. ' : : ' ' ' — ' 
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Rrinf Pf^^riptinn nf the Drawings 

. In the drawings, where like numerals refcjr to like components throughout the several 

views, ; r: _ , s ; 

Figure 1 shows a data warehouse decision support system according to the present 

invention; ; ^ t . — : ^ ... , ...... , . 

Figure 2 shows a more detailed implemCTtatioji of the data warehouse decision 

support system of Figure 1; , s : ^ : _ 

Figure 3 illustrates a star schema implantation of a ^ta warehouse according to the 

present invention;.? ^ t .. ;.: r v .: : Jv y\w;k .-.,-! ?} <> .• ! . iV ... ..v.; J; ; .. : , 

Figure 4 is a more detailed description, o£a .data warehouse according to the present 

invention; : \[ y ^ : .:\^:_ i W;,.., ^ -^ ;: : X: , ^ - . • . - - fl - ;: .;. : 

£igiire 5 shows an alternate ^bodim^ ^f.data warehouse and decision support 

syst^ according to, tte 

Figures . 6a and 6b illustrate distribution of adjusted data across levels in an unlocked 
and a locked system, respectively^ /t , ., > t ., : : ^t..-v . T . t • , . : „ 

r T)^rri ptior | the Pref^rrftH Fmhodirnents 

In the following detailed description of the preferred embodiments, reference is made 
to the accompanying dra\yings which fom apa^^CTepf, and in which is shown by way of 
illustration specific embodiments in which the inyentipn may be practiced. It is to be 
understood that other. ^embodiments may be, utilized^ and structural changes may be made 
without departing from the scope of the ?r?sent iRycartfon.^. ; ^ , , , 

Figure 1 illustrates a computer system 1 Ot haying m eito^ c^ to extract and 
modify data stored in a database. Computer system includes a memory 12 connected to a 
processor 14. Processor . 14 includes a cache 16, a reporting topi 19 and an override engine 
18. Memory 12 is used to store a database 20. Database 20 includes a plurality of data 
entries 26. . , . .-. ; -.; * ' 

.... In one embodiment, processor 14 also includes an instruction application process 34 
for permanently applying the modifications made within override engine 18 to data entries 26 
stored in database 20. In one; such embodiment, database 20 is a data warehouse and override 
engine; 18 is implemented as a multidimensional data entry/edit software engine process 
which supports the creation and adjustment of data points in the data warehouse. 
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In one embodiment, as is shown in Figure 2, cache 16 is implemented as a persistent 
cache which stores a subset of the plurality of data; entries 26. In one such embodiment, 
override engine 1 8 extracts data from the persistent cache for viewing by a user, modifies the 
data in response to one or more user commands and saves the user commands to an 
instruction file30 for laterapplication to database 20. 

A user sends commands to processor 14 over command interface 22 and receives 
reports based on data extracted from database 20 thrdugh report interface 24. In one 
embodiment, as is shown in Figure 2, command interface 22 and report interface 24 are 
implemented as a Single graphical^^ 1 

User action within system 10 is basically an adjustment process of data presenting 
future periods in time. In one embodimM^ 

systems and stored in the database 20. An example of this would be a statistically created 
forecast of future volume/ A^t^r cm th th£ Values (i:ei, facts) that 

they consider to be inaccurate. The combination^ the ^justaOTtxapabihty with the OLAP 
ability to present the data in ^ituMly siny l^dTbf jgrairalmty allbwS business professionals to 
review data within a familiar business context and use their kri6W16dge of the business to 
refine the data warehouse values. 

As with a standard read 6rily OtiAiP reporting syst^v ; Sy^em 10 allows end users and 
acfimnistrators' to ' define repoirte to ofg^mie anil present data. These ieports aie created by 
selecting thfe desired dimensional itifentxfi^ a&HVell ais the facts that contain the data needed 
to support the decision oir' 1 pi 3iu^ii^ pinciSD'cl^si " aft 6xariipfe, a'fepbit' may contain facts such 
as 4 Annual Plan/ 'Sta^ r " 
Region/ ^om Syrup,' and 'May 98; Jufie^98; atfia5uly^8? : " - : - ^ u: - 

All the data the user sees on the t^eport is stored in a work f He that was created for that 
adjustment cycle J The process 2^iniMstrktbr creates one work file for each adjustment cycle. 
For example, in a monthly planning cycle there would be one work file for the May 1996 
cycle, one for the June 1996 cycle, and so bn. . : 

When the user adjusts a fact value, the information is saved in a report file, not the 
work file. In the embodiment, when the final adjustment of the fact is completed, the process 
administrator updates the database with the new forecast data. 1 

As an example, the following chart sxunmarizes a typical adjustment cycle. In this 
case, a cycle used to create a consensus tactical forecast is shown; The chart identifies the 
required input, the steps^ and the results. L : ^ - ' ^ 
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Report files, tactical 


4. Forecasting administrator 


Management work file, 


work file. T 




management fact. 



As an; additional example in which the override 
application,; the following chart §\Hnmaroes a ectiy e management foirepasting cycle. It 
10 identifies the required input, the step^ and f th^ r fj . ; ^ 
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Override Process: Management Forecast .Cycle " ' 
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: Final business fact. 



'At the end of each cycfe, thfe process; ^^dministr^or prepared the d&ta'for the next 
cycie. Tliis involved swvtig the ' data* 'to ''the : <liit^baise^ ^fi^^xag the work file for the next : 
cycle, and setting up the single adjustable fact lor that cycle. The adjustable fact is the fact 
10 that can be changed or overridden. 

For example, at the end of the tactical forecast cycle, the Forecasting administrator 
updates the database with the consensus tactical forecast and sets up the next planning cycle, 
in this case a consensus management forecast. The following chart lists the steps. 
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5 A forecast override application such as system 10 allows users to apply qualitative 

methods (i.e., business knowledge) to the forecasting process. Users can change values 
generated by purely statistical methods to reflect changing business conditions. 

In one embodiment, system 10 maintains a master activity file. The master activity 
file consists of the full set of dimension information user information, and security 

1 0 information. It also contains detailed information about all of the facts that were chosen by 
the process administrator to support the planning; or adjustment process^ it is stored in only 
one place. 

Individual users have their own configuration files and the appropriate fact data files. 
When , the override application is started, the master data file is read and the entire dimension 
1 5 class is built in memory fox the user. The data structure has pointers to all of the data that the 
master structure points to. A system end user can view and modify any data that he has 
permission to see. The data that they are allowed to see can be configured oh a user by user 
basis to ensure that they see data they are authorized to access and in a usable context. 

A lock file is created while the work file is being processed by a user to ensure atomic 
20 usage of the data in the file. 

Reports 

In one embodiment, i reports are created from a template. The template is like a 
blueprint for the report. Each time a report is created, system 10 saves the blueprint. 

In piie emtodfanent, report templates are shared among users. Each user can then 
25 create their own version of the report For example, they can run a new report, modify the 
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report template, or u drilF to create a different version of the report. JBy drilling, they are able 
to review and adjust the data anew, with more discreet lev els of detail. As an example, 
instead of simply adjusting the fact 'Planned Budget* up by 25% at the annual level, the end 
user may 'drill down* to the monthly level and review and adjust specific months values. 
5 All reports contain one or more Columns and one 6r more rows. Each report can 

consist of multiple sections. ' If a report contains multiple sections, each section contains the 
same type of row and column information. 

The section, column, and row names in a report 'reflect the dimensions of database 20. 
These dimensions give meaning to the values 4&datafrase:20. It is, therefore, important to 
1 0 understand how the dimensions work together. 

Every data value in database 20 is defined by one or more dimensions. To fully 

describe each value, a report must contain at least one element from each of the available 

,-*"!* f.;.. . i '>''■-...:• : : '-rcn^ ; : >r u ■■ .. .. 

dimensions. 

• .'>\->*\'..-~:: ^fi^.-Xb-rrj c.' ^o--';; ! - '--J* ■■; ... : , \ v« ' : ■ 

Period dimensions are time intervals used for identifying and consolidating the data, 

15 such as weekly, monthly, quarterly, and yearly intervals. Non-period dimensions describe 
other aspects of your data. Non-period dimensions may include, for example, Geo-political, 
Product, and Business jOrg. A „ . .... 

Each dimension can consist of hierarchical levels. For example, product may be a 
dimension. Brand and SKU are associated product dimension levels. The levels represent 

20 differing degrees of detail and the paths used when you drill. 

: "' '■■-•-vv : ...!*ih-"i.--;;->f:.;;:.T»^: o.;r,- -* r ?t r\..-.y >. : ' „? V 

In one embodiment, database 20 is a data warehouse stored m one or more work tiles, 
.f ^^!X;: 1 u^::r^;!^^''':!.^;■: ( ^'J^^I -*;7 -a— s .<y — r " 
Each work file can contain an unlimited number of data points. (That is, the work file size is 

limited only by the amount of physical storage.) In one such embodiment database 20 is 

represented as a multidimensional database. Override engine 1 8 can modify and view data 

25 points at any combination of levels within the multidimensional database and, as values are 
changed, the new values are allocated to all levels of the dimension hierarchies. This ensures 
that summary levels of the data still total correctly. User instructions to the override engine 
1 8 are not applied to the data points. Instead the instructions are stored in instruction file 30. 
Since a single instruction may change tens of thousands of rows, just storing the instructions 

30 is much faster and less expensive than storing all the changed values. Once editing is 

complete the edits can be applied to the work file by executing instruction application process 
34. In this way, large scale data warehmise updates can be performed offline while the user is 
working on other tasks. 
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; By storing the instmctions rather than am 
10 can present the ; user with the results of the desired changes more quickly than in previous 
data warehouse analysis products. In addition, by using one or more work files rather than a 
predefined multidimensional database (MDDB) product, system 10 can handle data 

5 warehouses which are larger in size than current database products. 

In one embodiment, override engine 1 8 extracts the subset of data entries 26 from 
database 20 and stores the extracted data into cache 1 6. >£ Xn one such embodiment, override 
engine 18 employe the same mechamstn.used by reporAig tool 19 tp extract data from 
database 20. .For instance, in one such 9mtodmeiit, f both override engine 18 and reporting 

10 tool 19 determine t fo^ 

OLAP object ruiuiing on a standard request bpok^, v Sin$^^ override engine 18 and 

reporting tqol 1 9 look to a Metadata, table such as Metadata table 36 to identify fact tables, 
determine the levels? they need in each/act tabl^ ai>4 ^ extract the data from each required 
fact table. Override engine 18 then takes the report generated and pushes it into cache 16. 

15 From that point on* override engine 18, repeiyes,a c^mmmd, executes the command, and 
presents the data to the user. ( In addition, override j^gine 1 8,.stores the command in 
instruction; file 30 so that if pne .wants, to ren^&at^ report later, ^11 that has to happen is that 
the commands am re^plied to w 

* As .noted above, override engine 18 proyides the user or users ^yithja mechanism for 

20 quickly adjusting data stored in the data,^^eho^e dpring a subjwtiye-fact adjustment 
session, kon? ernbodimentj^daita stored in the dato warehp^e 
duringjsuch^ 

Commands to alter the data are stored to instruction file 34 and are used to change the data 
being displayed to the user;: The commands <?Q np^ .however, chmgethe data in. either cache 

25 16 or database 20. : ... . , -t^v:^ >r. v •> \ ; ' 

Data that's been stored back to the data, warehouse (i.e., stored in database 20) is 
"adjusted** or ''written" data. Data in database 20 is modified using the instructions stored in 
instruction file 34 via{a "commit** command. In one, such embodiment, data stored in 
persistent cache 1 6 is invalidated during execution of the "commit** command. 

30 Data which has been adjusted in cache 1 6 but which has not yet been written to 

database 20 is in that more nebulous state in the middle where it is adjusted but not 
committed. And that*Sithe situation where cache 16 does not match what's in the data 
warehouse anymore. In one such embodiment, the data gets to that state by executing a 
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**r^ieaie^ boiiunknd in engine '18. The "release" command causes override engine 18 to apply 
the commands not only to the dati presented to the screen but also to the data within cache 16 
itself, in one such embodiment, commands miist be "released" before the resulting changes 
are "committed" to database 20. Once a set of commands are Released", however, the 
5 original data is gone. ~ 

In another embcwiiinent, override engine 18 maintains a stack of old states and the user 
can scroll bkck through a list of releases to recovfcr an older state of cache 16. " J 
In one raibbdimeht, system 10 bnly eku^ts 6f the ', 

foretasting procies^ In subh m dribddime^^ bnly extracts data from database 20 

10 at the start 6f the f6rrca£tihjg process; In su&ilatt embodiment, Systein 10 applies a ; three-Step * 
process: work file generation, 'interactive Overriding, and database storage; < Work file ; :: 
generation is accoiiiplished xditig the fektract jinitse! Extract' reads the base statistical forecast 
data from database 20 and stor^ the data ifi i #brk ; Fife (cache 1 6)' in the UNIX file system 
of processor 14. - ^ ' ^ ;c * :n ' - ^ • - - ^' 

15 The remaining phases {dvemfle,; nhskA^^M <&mmit> fiisfe- the data in the Work file to 

generate adjusted fact <teta. System 10 allbws tbie useir to interactively change individual 
product 6r market values to reflect cHaiigmg business conditions. Each adjustment is 
allocated up and down the dimension liierai chy ^aggregation levels)- Allocations are 
perfdrined by using the dimension's drill hierifrchy to identify lower level components of the 
20 data point Adjusted These lbwdr leVel components are then given the new value that x - 

maintains their relative c^ritrtbutibn tb adjusted Value. Users f fnay also lock or unlock ^ 
individual values io that thb Values do hot ^ cha^ dti^g tiie adjustment process (directly or 
indirectly). 1 ' ; " ril fof f c : -^-->™<"' - - : - ' ' ' ; ' ■ ' ' ' 

Qnce*the adjuster has completed all wdfk for ah override, the adjustment is checked- 
25 in to the work file. The check-in process ensures that two adjustors do not adjust the same 
values and, in one embodiment, it allows a process administrator to review the work before 
commit. - : ; ' ' x ' 

The final phase of the override process is the database storage phase, or commit. 
Once the user or administrator is satisfied with the generated overrides, the overrides must be 
30 committed to the database. This is accomplished in the storage phase. 1 

As noted above^ extraction is the first phase of the override process. It is the 
generation of the work file. Extraction builds a snapshot of database 20 and stores the 
snapshot in the Work File in the UNIX file system of processor 14. Extraction is generally^ 
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done once because it is a time^nsuaning process and you generally do not jy/ant or need a 
user to see that happen. In one embodiment, the extraction process is performed by an agent 
executing within reporting tool 19. The agent will typically be set up to perform the 
extraction process after database 20 is refreshed. » >r ; 

5 In one embodiment, the extraction agent operates in response to user commands to 

extract the correct level of data from database 20. In one such embodiment, one could set up 
system 10 to respond to a command such as "I want these markets by these products for this 
period on the horizon, by channel" (or whatever your otiier dimensions are)- Another 
command syntax might be "Giye ; me all my pnwiucrts ;and giye ; me all my markets at the 

10 region level oyer a selected^ 

; In pne such t smbodiment, refresh, of ^^the^pe^istrat, cache is triggered by the current flag 
moving forwaM one mdnlh. ; For ex 

Februarys a subset of data isjread fron^da^j^e^ facts from database 

20 and calculated facts formed as a function p£d^ ^ 20 are stored in cache 16. 

15 Any data resident in pereistent cache 16^^ 

uncommitted* is.- simply overwritten* , : 0 i&r&.yj ?/:<; t , n ■ ; ry>- 

: To build the Work File, the system lOjmj^tdet^^ dimensions are drillable 

and at which aggregation levels the npn-drillable dimensions will be forecast. . In one 
embodiment, this information is stored in the Cate^^ } 

20 For drillable dimensions, data is stored in, the Work File at the .lowest Jeyel defined in 

the drill hierarchy for that dimension; For-non-^^ data is stored, only at the 

level of aggregation specified in the Metadata for ^atdjm^ipn; ^ , , r ^ : ft ,. : , 

The: Override phase allows the: user to ipt^actiYely adjust. the base statistical forecast 
data. Data is read from the Work File generat^by.the extraction phase. < The user is allowed 

25 to adjust values, lock and unlock values, drill upand down the drillable : diniensipn's: ... 
hierarchy, as well as many other functions. ; , 

Once an; adjuster has completed a set of override changes, these changes are released 
to the work file so that they are available for final commit to ; the warehouse. The release 
phase loads the base data from the work file, loads an instance file which contains the saved 

30 set of changes, and writes the hew values back to therwork file. 

The commit phase performs the commit for^the Override process. First, it reads the 
Work File. Next, fee data values of the lowest-level decomposed data are read from the 
warehouse. These values are used to properly proportion the data as it, is decomposed to the 
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lowestleveime^ 

written to the database. • 

For example, Suppose the data in the Work File is stored at the regional (REG) level 
of the Market dimension. Further suppose that the lowest level in me Market hierarchy is 
-5 market (MKT), Store will read th^REG level data from the work file, decortipose the REG 
level data to me MKT level, arid finally write the MKT level data to the database. 

The Database Structure '' " J: ■ v " " : ; : ' ; 3 ^ 

In one embodiment," databas^2d is m^iemetited as a nor^riorrrialized Star schema. A 
simplified version of a-siar scnOTia ^8Wshbwh ^erally iri Fig. 3* W a star schema, facts are 
10 stored as data m facVtabl^ multi-part key made up of the 

individual keys of each dimension; Snmrarly^d^ehsioh information 
tables: In the embodiment shown i in ^ Fig-^^^able 40 is a dimension table while the- 
tables labeled "Warehouse Ptodiicr* Warehouse Market"; "Warehouse Period" and o • > 
"Warehouse Fact" are fact tables 42; 44* 46 arid 48,'respectively, 
1 5 Non-normalized star schemas are designed for very fast data aggregation and < 

calculation: Such" speed can be veiy advantageous iri builditig the subset of data to be stored 
in cache' 16. Such sy store can, however, Bog ^wi : "ooiisideirably wb^-requved topeifoim * 
incremental updates (e.gl, as is the case during a forecastiri^^^^ * 
of cache 1 6 and instiuctibfr file BO-ar^'nistr^e^^wst^pottiiftg such forecasting sessions. 
20 ' : The down 

replication of data across tables/- AtiWe'same time^noweveri this replication of data, gi ve you 
the ability to ge : t your'keys fVc^ warehouse with' very small queries that database 20 

certainly can handle quickly: So a star scheriia [ approach is veiy well optimized for queries 
that pull a large number of rows i diit of database 20. • - ■ ■ . ; ' 

25 Standard OLAP reporting tools do not have to understand multiple levels.within a 

product or market hierarchy of a multi-tiered data warehouse. This is a key difference 
between override engine 18 and a standard OLAP reporting tool. Override engine48 must 
understand multiple levels at the same time because for the purposes of an adjustment, there 
may be interdependencies on the levels themselves for a reporting tool that essentially say, 
30 "O.K., here's data at this level," or "Here's data at the combination of these two levels." 

They're largely independent of each other and you can concentrate onion© or the other at any 
given time. Because of that, 6verride engine 1 8 does not rua directly against the warehouse. 
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Instead, oyerride.engine extracts that data, perfonns calculations where necessary and stores 
the modified d^ta to cache 16. r 

, : In .order to operate correctly, however, with database 20, override engine 18 and 
instruction application process 34 must understand the structure of database 20. In one 
5 embodiment, as is shown in Figure 3,. override engine 18 and instruction application process 
34 extract .fte stractme pf database 20 by,reviewing the contents of Metadata table 36. 

Such an approach pennits the use of an mJhmteji and easily alterable number of 
dimensions. In its simplest fpnri, metadata table 36 includes a period dimension table 40 and 
a fact table 46. Inoneemtodim^ 
10 description, field, a level column and a, hierarchy J^yel .^lijpim. In addition, period dimension 
table includes a.unique sequence number colump, a sequence within year, column and a 
current, period, column. Fact table 46 includes Jkeys wWch^e identical in type and structure 
to the keys listed in dimension table 40. . ,^> rt - . ^ . , 

For example, the metadata table can be used tp diiye a drill hierarchy tliat tells 
15 database 20 that days make up weeks and weeks make up months and months make up 

quarters, quarters make up halves, halves make up y^fs. ; Or that four quarters, make up a 
year, so that the user can jump and skip things^ Seething m the ^rodiict hierarchy. 
Override engine 18 and reporting engine 19 understand the Metadata structure and use the 
data stored in the dimension tables to extract data from database 20, ^ , ^ 
20 ) t f For example, And maybe 

it is category, manufacturer,, brands And then jqver in Mother tabte are lists of S>KUs by store. 

/ Another approach i? to ; haye a. separajt^ fagtt^ble for each month's data. That way 
when the next month'sdata arrive, it gets pl^^i^to a ney^ fact.tabje and stored in database 
20. An advantage of such an approach is that if one qf ^e.fact tabtes gete lost or corrupted, 
25 you can reload that month. In addition, as database 20 gets fei^er Md biggCT, its value to the 
company increases and so does the cost of maintaining Jhe database. Relational databases can 
scale into much larger data sizes and are much more maintainable than a corresponding multi- 
dimensional database. , r 

1 . It should be understood that database 20 may be distributed across a number of 
30 computers. In one embodiment, such as is shown in Fig. .5,. computer system 50 includes a 

server 52 connected to a processor 58 and a processor 62 by a network 56. In addition, server 
52 is connected to, a plurality 1 through N of workstations 62. Processor 58 and processor 62 
store portions of database 20 in memories 60 and 64, respectively. 
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In one embodiment, as is shoWn in Fig. 5, server 52 maintains a separate instantiation 
54 of override engine 1 8, reporting tool 19 and cache 16 for each user performing; tiie 
forecasting functibn. In another embodiment (not shown), a single cache 16 is used across all 
users. In either case^ changes to bache 16 are maintained in a separate instruction' file 30 for * 
- 5 each instantiation 52 and are only applied tothe ^ cache on receipt of a "release** command. 

In one embodiment, processor 58 is a multiple processor machine such as the Hewlett 
Packard ftt^OOO running ^'Oracle database application while processor- 62 is a Tandem 
Himalaya 1281 In one such embodiment; aggregated data is stored in the Oracle database on 
the HP machine wiiiie the lowest level daxa il fibred in the Himalaya machine?. The Metadata 
10 tells the program -the 'appropriate pn5c^5sof 'to WliiciiVqut^^shbuld be addressed; For 

example'/ if dita is stOT^ ~ 
determine this be looking it tiie Metadata. "If 1 , on &e other hand; the user is storing ^aggiegate 
data in cache 16, it will determine from the Metadata taftte that it should extiact'sucK dfdta 
from the Oracle database on processor 58: - ^ . , 
15 In another eambod^ i 8 arid reporting tool 19 ruri as a relatively 

thin windows client that essdhfially j&t p^d^GUI 32. All the analytics; all the storage, 
and all the real processing is done 6ri QNlX Server 52. ^ ' ?f 

locking Values in t he Database e; ; wi ' 

' In one ^bbdinient bverride engine iricMdes the ability to I6ck r values in database 20. 

20 As noted above,* oiia release, the chaiifees at 6rie level 6f thfe f hierarchy are pushed d&wn to 
each of tfte subleVelsof the Wdran&y' FW exaMp^ increased across the 

board by 10,000 uiiits, the id,o6o ! uiiils are ^Ssitfi^utfed proporiionaliy across ^ 
entities at the'ldWesflw^^ Ai B ' i — i r r - - 

In certain situations, such ari approaxili dbes not make sense. For instance, if one of 

25 three manufacturing plants is operating at capacity; it makes little sense to distribute the 

increase proportionally to the plant operating at capacity . For such situations, override engine 
1 8 includes a locking mechanism which can be used to lock the output of any of the 
manufacturing plants to a certain value. Any increases are then distributed proportionally 
across the remaining, unlocked, manufacturing plants. 1 ^ ;s; 

30 In one embodiment, the adjustment report displays the values of the dimensions the 

user selects on a Template dialog screen. The only values that can be adjiasfeil for the 



WO 99/57658 PCT/US99/09633 

16 r 

forecast are found in the unshaded cells. Ix^cked values are ^hfded red and non-adjustable 
values are shaded yellow; ^ r r r , 

The user can adjust a value at any level and the effects rippJ? through the rest of the 
product hierarchy . For example, if he or she adjusts a SKU yal^ie, the values of products at 
higher levels change to account for the lower leyel adjustment. < Or, if the user adjusts a higher 
level value, the values for the louver level products change j*s needed. 

.. - i Likewise, when the user locks a value, product values both up and down the hierarchy 
may be affected. For example, jfthe top; level is locked Vt 100, all lower values must total 

The diagram shown hvFig., 6a is t .m exapfd^ p^d^m^sitipn or proportipnal fitting . 
based on current forecast, data.. The current ^j^toient date is retrieved at manufacture level 
70 and, after an adjustment, is proportionally prop^at^ r t^xxgb Brand leyel J2 to SKU level 

If, however, the mMufacture product vaJUie .w^Jpc^ed.at.^yel 70,and one Brand 
product value was adjusted from 50, to 60, the adjustarat ^and its^effects are shown in Fig. 6b. 

; \ As noted above, in one embodiment both reporjting tool 1 9, and override engine 1 8 
have the ability to limit the eff^t pf con^ from dat^ase 20 

does not include the entire Merarchy for all dimensions. For instance, one m^y care about 
multiple levels of product dimension, r but would, like to limit the market adj ustments to a 
particular level (e.g., make them^et^justmen In ope such .embodiment, 

the user can select the level at >vhich data is adjusted. : For example,, pnp woyld tell override 
engine; 1 8 to/Ttfake^yo^ 

drillable, make your period dim^ippjipn^ must only be driven 

down the product hierarchy for display; the override engine dpes not haye to. try to drive it 
down the other hierarchies. As you add multiple drillable dimension^, die amount pf work 
that has to happen when you make adjustments expands exponentially. Not only does it go 
down product, but it has to go down product for every market* or every submarket, or every 
submarket of every submarket and you end up with a huge matrix of numbers to.keep track 

Drilling allows the user to display the fact values at different product levels, both up 
and down the product hierarchy. With drilling, one can see the ; aggregate values of a group of 
products; or can identify the specific values that went into the aggregate amount. 
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Dftiling up works with ariy'piotfiict where there are other products at a higher level. 
For example, one can drill up on a value if he or she is at SKU level 74 shown in Fig. 6a. 

Finally, a user can drill down with any product if he of she is at a level above the 
lowest level, SKU. Tor example, one can drill down if at manufacture level 70 in Fig. 6a. 

" If, for instance, you want to adjust the amount of a particular soft drink sold in a 
particular city, you could either make that ; a drillalile thing or a nori-drillable dimension. You 
can do it in one of Sfwo ways. One is you can set your market dimension as drillable, and then 
poll the whole market hierarcliy Sin ordfer ' td drill : down to the city ari<3 make* the change. If, 
however, you did not need that flexibility in drilling and the associated performance impact 
of allbcatiiig the 'changes through that <&m^^6n; ^bu coufd make the market dimension 
'nbn-drillable* and only see data at the city 1^1: ^en^<>uxj£m just grab whatever city you 
wanted arid make ybur adjus^ra^ 

level below city, once you do a "commit" it is going to drive the numbers to the bottom of all 
the hierarchies. (In one embtodinieht, r ^^^dela^itie l drives the numbers down to the bottom 
of all the hierarchies by modifying ^ch^u^tjier as a percent of contribution to the total.) 

For example, if given a ihaiket hierarchy including regions, cities within those regions 
and stores within those citids^ybu Wei£ going i r td 'modify tihife'at the -regional level, you only 
need to maintain data in cabhe 16 at that levels To do this, Override Engine 18 initiates a 
transfer of data at the atomic 16veU receives ttid^ data and ! pushes it up to the r^gSohalleve) . 
The aggregated regional data is then itored iri^ 

applied against ^at ag^r^gdte date/ Then; v/fie&ybu arertady to* store the changed data, a 
"coniroit 1 ' coirtmm^ atomic level in 

database itii The result is tliat usCTcbii^^ and 
get stored k^ii^tely ttie ; komic fev&r' * ; ■ : ^ r 

Fot example, one might deteihiihe Whaft : would happen if volume incr eased by 10% in 
a region such as the Great Lakes territory. The change would be made at the territory level 
and, when committed, it is distributed to the cities as a function of the percentage they 
contribute to the regional total/ 

No matter the level where the adjustment is made, once the warehouse is updated, it is 
updated at the lowest atomic level and then in this case for reporting engine 19, then it can 
automatically pick up a report of it based on market hierarchy, period hierarchy, etc. All that 
is a trade-off between speed and flexibility! You can go ultimately flexible if -you've got 
enough iron behind the thing to drive it. And if you do not, then you can cut back. 
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Also from a security perspective, quite often if it is .sales people that are using it for 
the adjustments, they do not want them to see other territories, for instance, other people's 
clients, other sales, guys' clients. Otherwise they start playing games that you do not want 
them playing. It is a combination of those? things. 

Override engine 18 understands all of reporting tool 19*s filters and calculated facts; 
it also understands the Werarchies of system 10. Essentially everything in the Metadata that 
can be defined, is read by reporting tool 19 and override engine 1,8 ancjthey will use those 
definition. So.you can create a,calculated fact in r^ortfnglopl l^and override engine 18 
will automatically see it and know how ^tp use it. ^ ^ ^ ^ , ; i ; i 

(An example of a/calculated factwpul&be gross revenue, Gross revenue is calculated 
as units times price., YouiWQuld not, ther^foreistor^ SFP$$ I ? stea ^» ypu qalculate it 

off of the two that you did store- (price and^te);^ABO^ eiror. 
Facts calculated as a functiofn of an actual fact ^^ r ^x^^X^ fac\ ar^ derived, not stored. 
Override engine 18 autom^cally readsvthe ^^tion^qfjeach calculated fact out of the 
Metadata and applies them on the fly to the data read froni database 20.) 

In one embodiment, forecasts are stored ^separate databases ; within database 20. For 
instance, one c»uld include a dimension !ate^ 

therefore, be critical to have the ability to add extra dimensions to systems 10 and 50. 

••'■< ' ' — <' * ' ^f 1 ■ ■ ' * /ii i .-• tr, , " ;;. ... \- 

As was noted in connection with Figure 2, in one embodiment cache 16 is a persistent 
cache stored as a B-tree in a UNIX file system on processor 14. The B-tree allows you to 
essentially go with a much more compact data storage in situations where the matrix of data 
is populated sparsely (such as in, for example, customer centric databases) yet, at the same 
time, a B-tree implementation does allow you to get reasonably fast access times. 

In one such embodiment, the persistent cache is implemented using standard Rogue 
Wave B-tree code. The performance of the Rogue Wave B-tree code can be enhanced by 
replacing the standard I/O stream (which is write character by character) with an I/O stream 
which writes data in large blocks of data. 

In addition, with a B-tree implementation, one can trade off size for speed by 
increasing or decreasing the number of branches and subbranches in the tree. That is, the 
deeper you make the B-tree, the slower your access time but, at the same time, the smaller the 
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actual physical storage. In orie embodiment, the structure of the B-tree is optimized for size 
overspeedl 

In more densely padked matrices of data, cache 16 cari be implemented as a set of file- 
based arrays. Such arrays are very fast, requiring only simple calculations to get to the right 
locations. They can be, however, very inefficient space-wise. Typically, if one of the 
dimensions of the data warehouse is customer, and the number of potential customers is large, 
you are better off selecting a B-tree implementation for cache 16. z { 

Override engine 1 8 allows the iiiSr to grab slices of data from a database that may be 
too big to comprehend in its entirety. Ovemde^hgihe 18 allows you to slice a portion of the 
database but, which sometimes in■A(B : i^dii^ , ^l^ , l^rtIw to as a datamark,; automatically 
and intelligently through the liser of nietadat^? : The portion extracted can be dealt with either 
within at outside th6 "databW^'quidkiy ^d-effiB^tly'. 1 : 'Oiice'thg^hahgeis--are in place, / : 
override engine 18 automatically md r ^ 

result is a database tool which eliminates 11^ pfery traffic bottleneck of traditional approaches 
to relational database '-management systebi& ' : < 

Although specific embodiments have been illustrated and^ described Imrein, it ^ 
appreciated by those of ordinary ski ll in the art that any arrangement which is calculated to 
achieve the serine ptiipbse may be isubstituted for the specific embodiment shown. ; This; 
application is intended to cover any adaptations or variations of the present invention. 
Therefore, it is intended that this invention be limited only by the claims and the equivalents 
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What is claimed is: 

1. A data warehouse, comprising: 

memory, wherein the memory includes a database having a plurality of data entries; 
a processor connected to the memory, wherein the processor includes: 

a cache; and , - , , :>.. t .- r - 

an override engine; 

wherein the cache includes a subset of the plurality of data entries and wherein the 
override engine extracts data from the cache for .viewing by a user, modifies the data in 
response to, one or more tiser commands and sa^es the user commands to a file, for later 
application to the database. t . v , 

2. A data warehouse, comprising: ^ : , M , 

memory, wherein the memory incites ^ ; d§^ of data entries; 

a server connected to the memory, wherein the processor includes: 
... a cache; andk- r; -J 

an override engine; and * ... : 9:; . ; , , - t . s ; : 

/ a client work^tion=conn€^ted to.the server; :t . :\ v . 
wherein the server operates in response to user commands to store a subset of the . 
plurality of data entries in the, cache v wherein the override engine extracts data from the cache 
and sends the data extracted from the cache4o the- client workstation for viewing by a user, 
modifies the data in response to one or more of th§jUser commands, saves the user commands 
to a file and operates in response to a commit command to modifies the database based on the 
user commands stqi^ hi theu^^ t . . r ■• 

3. A method of reporting data from a data warehouse,; the method comprising the steps 
of: 

providing a server and a memory device- 
storing a database in the memory device, wherein the database includes a plurality of 
data entries; 

extracting a subset of data entries from the database; 
storing the subset of data entries on the server, : : 

modifying the data entries stored on the server in response to user commands; 
reading data from the modified data entires stored on the server; and> 
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displaying the data to the user. 

4. A method of forecasting, comprising the steps of: 
providing a server and a memory device; 

storing a database in the memory device, wherein the database includes a plurality of 
data entries; 

exttacting a subset of data entries' firom the database; ' - * : • v - 

storing the subset of dali entries ofi the stiver; ; r \. z 

modifying the data entries stor^ bn ^ v 

storing the user commands; . t ; ) - ^ 

reading data from the modified data entries stored on the server; 

displaying the data to the user; and v : ^ :^ r ;^ :- ^. t ; -j 

modifying the database basedxa " ' ' v *■* 

5 . A method of increasing the speed in which changes to a relational database are 
reflected back to the user, comprising the steps of: : 1 ■ - 

extracting a subset of data from the database, wherein the step of extracting includes # 
the steps of: ; * " : " '-" *" "' ' ; ; • ■ ■ - - - - ; \- 

■ L displaying k representation of the subset 6f data to the usCT; ^d ^ - ^ M 
v *" / storing the subset of data iri a-ciache; - - 1 lf - ; : ; ; • ^ : - 
J 'recaving at data modificati^ * . ^ ^ ; ^ r , ^ ? ; • 

r storing tl^ data m^ ^ ^ ■ ' ; v,/ * : c 1 

applying the data modification command against the subset of datk stbred in the 
cache, wherein the step of applying the data modification command includes the step of 
modifying the subset of data to reflect replication of the data modification command. 

6. The method according to claim 5, wherein the step of applying the data modification 
command further includes the step of modifying the representation of the subset of data to 
reflect application of the command. 

7. The method according to claim 6, wherein the step of applying the command further 
includes the steps of: - ' > v . v ■ 

waiting for a "commit" command; and r: 
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on receipt of the "commit" command, modifying data in the database to reflect 
application of the data modification command. 



8. The method according to claim 5, wherein =the step of applying the command further 
includes the steps of: « 

waiting for a "commit'* command; and 

on receipt of the "commit* * commandj modifying data in the 
application of the data modification command; 



database to reflect 



9. A data structure for updating a database, comprising: 

a plurality of user data-modification commands, wherein the plurality of user data- 
modification commands provide information for modifying information in a database (20). 

10. The data structure of claim 8, wherein: : 

the plurality of user data-modification commands provide data-modification 
information for application tb aggregated regional data stored in a cache (16), and 
modification information that gets drilled down to in atomic level in the database (20). 
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